In order to generate a stereoscopically visible image with a stereo camera (stereo imaging), it is desirable that the focal distances of the two cameras or the optical positions such as the attachment positions are accurately aligned with each other. However, in practice, the optical positions of both cameras are not always aligned, and therefore, two images taken by both cameras (stereo image) include display error due to the difference between the optical positions of the stereo camera. Therefore, an optimum stereoscopically visible image that provides a depth feeling or a protrusion feeling cannot be displayed unless the error is corrected so as to appropriately adjust the display positions of both images (stereo image).
In addition, in conventional correction methods, an image for adjustment (adjustment pattern) is to be prepared. Therefore, even though distributers may be able to perform such adjustment (correction) in manufacturing or shipping, it is difficult for a user to perform the adjustment by using that method. Meanwhile, owing to outer shock to the apparatus when used, aging, or the like, the installation position (optical position) of the stereo camera can change. In such a case, a correction method for a user to easily perform the adjustment is desired to be provided.
Therefore, a feature of the exemplary embodiments provides a computer-readable storage medium having stored therein information processing program, and the like that allow a user to easily adjust the display positions of two images composing a stereo image even if the optical position of an imaging section (stereo camera) has changed owing to outer shock to the apparatus when used, aging, or the like.
The exemplary embodiments have the following aspects in order to solve the above problems.
One aspect of the exemplary embodiments is a computer-readable storage medium having stored therein an information processing program which is executed by a computer of an imaging apparatus that takes an image of a subject by a stereo imaging section to acquire a stereoscopically visible taken image and that includes a storage section. The information processing program causes the computer to function as: a temporary taken image acquiring section, a temporary taken image display section, an input accepting section, a display position changing section, a correction value calculation section, and a storage control section. The temporary taken image acquiring section takes an image of a given imaging target by the stereo imaging section, and acquires an image taken by a first imaging section of the stereo imaging section, as a first temporary taken image, and an image taken by a second imaging section of the stereo imaging section, as a second temporary taken image. The temporary taken image display section displays the first temporary taken image and the second temporary taken image on a display section. The input accepting section accepts an input from a user. The display position changing section changes at least one of the display positions of the first temporary taken image and the second temporary taken image, based on a predetermined input accepted by the input accepting section. The correction value calculation section calculates a correction value for correcting at least one of the display positions of images taken by the first imaging section and the second imaging section on the display section, based on displacement, by the changing, of the at least one of the display positions of the first temporary taken image and the second temporary taken image that has been changed by the display position changing section. The storage control section stores, in the storage section, the correction value calculated by the correction value calculation section without associating the correction value with the first temporary taken image and the second temporary taken image.
According to the above aspect, adjustment is performed based on the first temporary taken image and the second temporary taken image (adjustment images) acquired by taking images of a given imaging target (subject) with a stereo camera composed of the first imaging section and the second imaging section. Therefore, a user can acquire adjustment images by taking images of a desired imaging target with the stereo camera, and does not need to separately prepare a pattern image of the like for adjustment. In addition, based on a predetermined input from a user (a user's operation for moving at least one of the adjustment images), the display positions of the adjustment images displayed on the display section (for example, the display screen of the stereo camera) are changed. Therefore, a user can move the adjustment images to a desired position while confirming the display screen. In addition, based on displacement of the display position of the adjustment image changed based on an operation by a user, the correction value for correcting the display position of a stereo image (composed of two images taken with the stereo camera) to be displayed in a stereoscopically visible manner on the display surface is calculated, and the calculated correction value is stored in the storage section. Therefore, a user only has to do simple image adjustment (move adjustment images), in order to calculate the correction value for appropriate adjustment of the display position of a stereo image and store the correction value in the storage section.
In another aspect, the computer may be caused to function as a correction section and a stereoscopic display control section. The correction section, based on the correction value, may correct the display position, on the display section, of at least one of a first image taken by the first imaging section and a second image taken by the second imaging section which are taken separately from the first temporary taken image and the second temporary taken image. The stereoscopic display control section may display the first image and the second image on the display section in a stereoscopically visible manner, based on a correction result by the correction section.
According to the above aspect, based on the correction value, the display positions of the first image and the second image (stereo image) on the display section (display screen) are corrected, and based on the correction result, the stereo image is displayed on the display screen in a stereoscopically visible manner. Therefore, a user only has to do simple image adjustment (move adjustment images), in order to appropriately adjust the display position of a stereo image and appropriately display the stereo image on the display screen in a stereoscopically visible manner.
In another aspect, the calculation by the correction value calculation section may be performed as follows. That is, the correction value calculation section may calculate the correction value, based on only displacement in the vertical direction of the at least one of the display positions of the first temporary taken image and the second temporary taken image.
According to the above aspect, although displacement by movement of the display position of the first temporary taken image and/or the second temporary taken image (adjustment images) includes displacements in various directions, the correction value is calculated based on only displacement in the vertical direction (the y-axis direction shown in FIGS. 1, 5, and the like described later) of the adjustment images. Thus, the correction value is calculated as a value for correcting displacement of the position of a stereo image (displacement in the vertical direction between the positions of the first image and the second image) that is a main cause of recognition of the stereo image as two images displaced from each other. In addition, if the adjustment images can be moved also in directions other than the vertical direction by a user, it becomes possible to substantially overlap the first temporary taken image and the second temporary taken image with each other. As a result, it becomes easy to align (adjust) the display positions of the first temporary taken image and the second temporary taken image with respect to the vertical direction and to calculate appropriate correction value.
In another aspect, a display parameter for determining the display positions of the first image and the second image on the display section may be stored in the storage section. In this case, the temporary taken image display section may display the first temporary taken image and the second temporary taken image on the display section, based on the display parameter. The correction section may correct at least one of the display positions of the first image and the second image on the display section, based on the display parameter corrected based on the correction value.
According to the above aspect, the display parameter for determining the display position of a stereo image (first image and second image) to be displayed on the display section (display screen) is stored in the storage section (for example, a main memory). Based on the display parameter, the display position of a stereo image is determined. The display position of a stereo image is corrected by the display parameter being corrected. Therefore, a user can cause a stereo image to be displayed at an appropriate display position on the display screen, in which the correction is reflected (i.e., at the position updated based on the correction).
In another aspect, the computer may be caused to function as a display position determination section and a display adjustment parameter calculation section. The display position determination section may determine at least one of the display positions of the first temporary taken image and the second temporary taken image, based on a determination input accepted by the input accepting section. The display adjustment parameter calculation section may calculate a display adjustment parameter for correcting at least one of the display positions of the first image and the second image on the display section. In this case, the storage control section may update the display parameter to the value of the display adjustment parameter, and stores the updated display parameter in the storage section. The display position changing section may change the display position of the first temporary taken image, based on a first input accepted by the input accepting section. The display position determination section determines the display position of the first temporary taken image to be that changed by the display position changing section, based on the determination input. The display adjustment parameter calculation section may calculate the display adjustment parameter, based on a first displacement which is a displacement between the display position of the first temporary taken image determined by the display position determination section, and the display position of the first temporary taken image as it was before the changing by the display position changing section. The correction section may correct at least one of the display positions of the first image and the second image on the display section, based on the updated display parameter stored in the storage section.
According to the above aspect, if a user performs an operation of indicating that the display position of the first temporary taken image has been aligned with the display position of the second temporary taken image (the adjustment has been completed) (that is, if a user has given a determination input), the display adjustment parameter is calculated based on movement information (information about displacement of the display position) about the first temporary taken image (one of the adjustment images) moved thus far. Then, the display parameter, which determines the display position of a stereo image (first image and second image) to be displayed on the display section (display screen), is updated to the value of the display adjustment parameter. Therefore, a user only has to align (adjust) the display positions of the first temporary taken image (one of the adjustment images) and the second temporary taken image (the other one of the adjustment images) while confirming displacement of the first temporary taken image (the one of the adjustment images) on the display screen, whereby the display parameter can be updated (corrected) to an appropriate value (display adjustment parameter), and a stereo image can be displayed at an appropriate display position based on the updated display parameter.
In another aspect, the changing of the display position by the display position changing section, the determination of the display position by the display position determination section, and the calculation of the display adjustment parameter by the display adjustment parameter calculation section, may be performed as follows. The display position changing section may further change the display position of the second temporary taken image, based on a second input accepted by the input accepting section. The display position determination section may determine the display positions of the first temporary taken image and the second temporary taken image to be those changed by the display position changing section, based on the determination input. The display adjustment parameter calculation section may calculate the display adjustment parameter, based on the first displacement, and a second displacement which is a displacement between the display position of the second temporary taken image determined by the display position determination section, and the display position of the second temporary taken image as it was before the changing by the display position changing section.
According to the above aspect, the first temporary taken image and the second temporary taken image (both adjustment images) can be moved by a user's operation (first input and second input). Therefore, a user can align (adjust) the display positions of both adjustment images by moving both adjustment images. In addition, the display adjustment parameter is calculated based on movement (displacement) information about both adjustment images. Therefore, a user can more easily align the display positions of the adjustment images than a user aligns the display positions of the adjustment images by moving only the first temporary taken image (one of the adjustment images). Based on the adjustment result, a stereo image can be displayed at an appropriate position.
In another aspect, the calculation of the display adjustment parameter by the display adjustment parameter calculation section may be performed as follows. The display adjustment parameter calculation section may calculate the display adjustment parameter, based on a relative displacement of the first displacement with respect to the second displacement.
According to the above aspect, while the first temporary taken image and the second temporary taken image (both adjustment images) can be moved by a user's operation (first input and second input), relative displacement of the first temporary taken image with respect to the second temporary taken image (displacement of one of the adjustment images from the other one) is used for calculation of the display adjustment parameter. Therefore, when a user aligns the display positions of the adjustment images, the user can more easily align (adjust) the display positions of the adjustment images by moving both adjustment images. In addition, the calculation of the display adjustment parameter in this case can be performed similarly to the calculation performed when the second temporary taken image is fixed and only the first temporary taken image is moved (that is, only one of the adjustment images is moved). Therefore, the display adjustment parameter can be easily calculated.
In another aspect, the calculation of the display adjustment parameter by the display adjustment parameter calculation section may be performed as follows. The display adjustment parameter calculation section may calculate the display adjustment parameter, based on only a displacement in the vertical direction of the first displacement.
According to the above aspect, although displacement (the first displacement) by movement of the display position of the first temporary taken image includes displacements in various directions, only displacement in the vertical direction (the y-axis direction shown in FIGS. 1, 5, and the like described later) is used for calculation of the display adjustment parameter. Thus, displacement in the vertical direction of the position of a stereo image (the first image and the second image) that is a main cause of recognition of the stereo image as two images displaced from each other, is corrected. In addition, while only displacement in the vertical direction is used for calculation of the display adjustment parameter, if the first temporary taken image (one of the adjustment images) can be moved also in directions other than the vertical direction by a user, it becomes possible to substantially overlap the first temporary taken image and the second temporary taken image with each other. As a result, it becomes easy to align (adjust) the display positions of the first temporary taken image and the second temporary taken image with respect to the vertical direction.
In another aspect, the calculation of the display adjustment parameter by the display adjustment parameter calculation section may be performed as follows. The display adjustment parameter calculation section may calculate the display adjustment parameter, based on only a displacement in the vertical direction of the first displacement and a displacement in the vertical direction of the second displacement.
According to the above aspect, although displacement (the first displacement and the second displacement) by movement of the display position of each of the first temporary taken image and the second temporary taken image includes displacements in various directions, only displacements in the vertical direction (the y-axis direction shown in FIGS. 1, 5, and the like described later) of the temporary taken images are used for calculation of the display adjustment parameter. Thus, displacement in the vertical direction of the position of a stereo image (the first image and the second image) that is a main cause of recognition of the stereo image as two images displaced from each other, is corrected. In addition, while only displacement in the vertical direction is used for calculation of the display adjustment parameter, if the first temporary taken image and the second temporary taken image (both adjustment images) can be moved also in directions other than the vertical direction by a user, it becomes possible to substantially overlap the first temporary taken image and the second temporary taken image with each other. As a result, it becomes easy to align (adjust) the display positions of the first temporary taken image and the second temporary taken image with respect to the vertical direction.
In another aspect, the storage control by the storage control section, the calculation of the display adjustment parameter by the display adjustment parameter calculation section, the stereoscopically display control by the stereoscopic display control section, may be performed as follows. The storage control section may store, in the storage section, the display position of the first temporary taken image changed by the display position changing section, every time the input accepting section accepts the first input. When the input accepting section has accepted a switching input, the display adjustment parameter calculation section may calculate the display adjustment parameter, based on a third displacement which is a displacement between the display position of the first temporary taken image stored in the storage section, and the display position of the first temporary taken image as it was before the changing by the display position changing section; and the stereoscopic display control section may display the first image and the second image on the display section in a stereoscopically visible manner, based on the calculated display adjustment parameter.
According to the above aspect, every time the display position of the first temporary taken image (one of the adjustment images) is changed by a user's operation, the changed display position is stored in the storage section, the display adjustment parameter is calculated based on the stored display position, and a stereo image (first image and second image) is displayed in a stereoscopically visible manner, based on the calculated display adjustment parameter. Therefore, when a user is adjusting the display position of the first temporary taken image (one of the adjustment image) (that is, during the adjustment), the user can actually confirm how the stereo image is displayed in a stereoscopically visible manner by the current adjustment, and can confirm whether or not the current adjustment is appropriate, before ending (determining) the adjustment.
In another aspect, the storage control by the storage control section, the calculation of the display adjustment parameter by the display adjustment parameter calculation section, the stereoscopically display control by the stereoscopic display control section, may be performed as follows. The storage control section may further store, in the storage section, the display position of the second temporary taken image changed by the display position changing section, every time the input accepting section accepts the second input. When the input accepting section has accepted the switching input, the display adjustment parameter calculation section may calculate the display adjustment parameter, based on the third displacement and a fourth displacement which is a displacement between the display position of the second temporary taken image stored in the storage section, and the display position of the second temporary taken image as it was before the changing by the display position changing section; and the stereoscopic display control section may display the first image and the second image on the display section in a stereoscopically visible manner, based on the calculated display adjustment parameter.
According to the above aspect, every time each of the display positions of the first temporary taken image and the second temporary taken image (both adjustment images) is changed by a user's operation, the changed display position is stored in the storage section, the display adjustment parameter is calculated based on the stored display positions, and a stereo image (first image and second image) is displayed in a stereoscopically visible manner, based on the calculated display adjustment parameter. Therefore, when a user is adjusting the display position of the first temporary taken image and the second temporary taken image (both adjustment images) (that is, during the adjustment), the user can actually confirm how the stereo image is displayed in a stereoscopically visible manner by the current adjustment, and can confirm whether or not the current adjustment is appropriate, before ending (determining) the adjustment.
In another aspect, display of the temporary taken image by the temporary taken image display section may be performed as follows. The temporary taken image display section may display the first temporary taken image and the second temporary taken image on the display section such that at least one of the first temporary taken image and the second temporary taken image is displayed in a translucent manner.
According to the above aspect, the first temporary taken image and the second temporary taken image (both adjustment images) are displayed on the display section such that at least one of the temporary taken images is displayed in a translucent manner. Therefore, a user can easily view displacement between the two images. In addition, since the adjustment images have increased visibility, a user can intuitively and easily align (adjust) the display position of the first temporary taken image (one of the adjustment images) with the display position of the second temporary taken image (the other one of the adjustment images) by changing the display position of the first temporary taken image.
In another aspect, display of the temporary taken image by the temporary taken image display section may be performed as follows. The temporary taken image display section may display the first temporary taken image and the second temporary taken image on the display section in a planar visible manner.
According to the above aspect, the first temporary taken image and the second temporary taken image (both adjustment images) are displayed on the display screen in a stereoscopically visible manner (that is, the images are displayed such that both images are viewed by each of the right eye and the left eye of a user). Therefore, a user can easily view displacement between the display positions of the adjustment images, and can easily adjust these images.
In another aspect, a default parameter that defines predetermined default display positions of the first image and the second image on the display section may be stored in advance in the storage section. In this case, when the input accepting section has accepted a defaulting input, the storage control section may update the display parameter to the value of the default parameter and store the updated display parameter in the storage section.
According to the above aspect, if a user performs an operation of defaulting the display position (operation of returning the display position of a stereo image to the default setting), the display parameter is changed to the value of the default parameter. Therefore, even when a user has changed (adjusted) the display position of a stereo image, the user can return the display position to the default setting any time.
In another aspect, the changing of the display position by the display position changing section may be performed as follows. The display position changing section may change, in the horizontal direction and the vertical direction, at least one of the display positions of the first temporary taken image and the second temporary taken image, based on the predetermined input.
The above aspect is effective particularly when the first temporary taken image and the second temporary taken image (the adjustment images) are displayed in a planar visible manner. In general, since stereoscopically visible images have parallax therebetween, if the adjustment images are displayed in a planar visible manner, the adjustment images are displayed being displaced from each other in the horizontal direction (the x-axis direction shown in FIGS. 1, 5, and the like described later). Therefore, even if displacement between the adjustment images due to the optical position of the stereo camera includes only displacement in the vertical direction (the y-axis direction shown in FIGS. 1, 5, and the like described later), the adjustment images are displayed being displaced also in the horizontal direction. The magnitude of the displacement in the horizontal direction varies depending on the distance between an imaged subject and the stereo camera. Therefore, it may be difficult for a user to perform adjustment for removing displacement in the vertical direction. However, according to the above aspect, since at least one of the adjustment images can be moved also in the horizontal direction, a user does not need to consider the distance between a subject and the stereo camera, and can cause the adjustment images to substantially overlap with each other by moving at least one of the adjustment images in the vertical direction and the horizontal direction. Thus, a user can align (adjust) the display positions of the adjustment images with respect to the vertical direction.
In the above, the exemplary embodiments have been described using a computer-readable storage medium having stored therein an information processing program as an example. However, the exemplary embodiments may be applied to an imaging apparatus, an imaging system, or an imaging method.
According to the exemplary embodiments, it is possible to provide a computer-readable storage medium having stored therein information processing program, and the like that allow a user to easily adjust the display positions of two images composing a stereo image even if the optical position of an imaging section (stereo camera) has changed owing to outer shock to the apparatus when used, aging, or the like.
These and other objects, features, aspects and advantages of the exemplary embodiments will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.